1. Field of the Invention
The present invention relates to an image processing method, an image processing apparatus, a non-transitory computer-readable medium, and an image-pickup apparatus.
2. Description of the Related Art
An image captured through a wide-angle lens, an image captured by an image-pickup apparatus having a small image sensor size, and an image captured by an image-pickup apparatus having a small aperture diameter or a large F value has a deep depth of field with few blurs from a short-distant view to a long-distant view. In order to emphasize the main object and to relax the atmosphere, there have been proposed a variety of methods of blurring a captured image.
For example, Japanese Patent Laid-Open No. (“JP”) 2000-259823 proposes a method for obtaining distance information at a plurality of focus detection points for auto focusing (AF) and for blurring an input image. JP 11-41512 proposes an averaging method of pixel values in a user specified area in order to blur an input image. A. Rosenfeld (author), Makoto Nagao (translator), “Digital Picture Processing,” Kindai kagaku sha Co., Ltd. (1978), proposes a variety of image processing methods of correcting a deteriorated input image. Other prior art contain Ren Ng et al., “Light Field Photography with a Hand-Held Plenoptic Camera,” Stanford Tech Report CTSR, February of 2005.
JP 2000-259823 and 11-41512 discuss methods of blurring the input image based upon the pixel value of the input image, but the pixel value of the input image is influenced by the optical characteristic of the image-pickup optical system regardless of the size of the image sensor and the focal length. An image shape formed on the image sensor is different from the object even at the in-focus position. At the in-focus position, the object in the periphery of the image is asymmetrically deformed due to the coma and the astigmatism. The image also deforms due to the diffraction. At a defocus position, the image deforms due to the vignetting in addition to the aberration and the diffraction. When a blur is added to an image that has deformed due blurring of the aberration, blurring of the diffraction, and blurring of vignetting, a blurring shape is unexpectedly different from one supposed by the object shape.
For example, if a circularly symmetrical blur is added to a captured image (input image) of an object illustrated in FIG. 19A, the blurring shape is as expected from the object shape, as illustrated in FIG. 19B. However, when the captured image is influenced by the coma, the captured image of the object illustrated in FIG. 19A deforms as illustrated in FIG. 19C. If circularly symmetrical blurring is applied to this image (the image of FIG. 19C), the blurring shape is subject to the coma as illustrated in FIG. 19D.
If the captured image is influenced by the astigmatism, the captured image illustrated in FIG. 19A deforms as extending in a specific direction. The image at the defocus position deforms due to blurring of the aberration and blurring of vignetting, for example, into an image lacking of part in the image illustrated in FIG. 19A. In other words, in order to obtain a good blurring shape as expected from the object shape, it is necessary to restrain image deformations caused by blurring of aberration, blurring of the diffraction, and blurring of vignetting.
FIG. 20A is a graph illustrating the partially saturated brightness in an input image. In each of FIGS. 20A to 20C, an abscissa axis denotes a space coordinate and an ordinate axis denotes a brightness value. Herein, the saturation value is defined as a maximum brightness value that can be expressed in a digital image. If a blur is added to the brightness distribution of the image illustrated in FIG. 20A in which the brightness is partially saturated in the input image, a curve of the brightness distribution will be milder as illustrated in FIG. 20B than that of FIG. 20A. FIG. 20C illustrates an image obtained with a smaller F value (or under a brighter condition) than that of the image of FIG. 20A in order to blur an image with a shallow depth of field. The image illustrated in FIG. 20B is darker than that of FIG. 20C because the brightness is reduced. In other words, blurring cannot be well reproduced in an area of the brightness saturated part in an image captured with a small F value simply by blurring the input image.